The present invention relates to methods and apparatuses for reducing the amount of data needed to represent an image, and more particularly, to a novel method and apparatus for compressing motion pictures.
Images are conventionally represented by a two-dimensional array of values in which each value represents a property of the image at a corresponding point on the image. In the case of gray-scale images, a single number representing the gradations of intensity from white to black, referred to as the gray scale, is stored. In the case of color images, each "value" is a vector whose components represent the gradations in intensity of the various primary colors, or some alternative color code, at the corresponding point in the image.
A motion picture comprises a sequence of such images. Typically, thirty or more images are needed for each second of viewing time.
This representation of a motion picture corresponds to the output of a typical image-sensing device such as a television camera. Such a representation is convenient in that it is easily regenerated on a display device such as a CRT tube. However, the number of bits needed to represent the data is prohibitively large for many applications. A single 512.times.512 gray-scale image with 256 gray levels requires in excess of 256,000 bytes. At 30 frames per second, a communication channel with a bandwidth of approximately 64 million bits per second is needed to transmit the motion picture. A full color 24 bit per pixel motion picture would require a bandwidth of more than 190 million bits per second.
This bandwidth is significantly greater than that available for many communications purposes. For example, high quality consumer telephone lines are typically limited to 64 thousand bits per second. Hence, commercially viable picture telephone systems require some form of image compression.
Image compression systems used in motion picture compression applications make use of the redundancy between frames to reduce the amount of information that must be transmitted over the bandwidth-limited communication link. Many scenes in a motion picture include portions that are constant from frame to frame for several seconds, if not minutes. Prior art compression schemes attempt to divide the scene into the constant portion and the portion that changes from frame to frame.
The constant portion need be sent only once. Hence, if the constant portion comprises a significant fraction of the scene, considerable image compression can be realized. For example, if the entire scene were constant for 10 seconds, the information needed to reproduce the scene would be contained in one frame of the sequence and an indication of the number of times the frame is to be repeated. This would be approximately 1/300 of the information needed if the scene were sent without compression.
To better understand the manner in which this type of prior art image compression operates, consider a motion picture which is being sent over a bandwidth-limited communication link from a transmitter to a receiver. The receiver will be assumed to have a frame buffer which stores one frame of the motion picture. The contents of this frame are then displayed at the receiver. Ideally, the frame in the receiver will exactly match the corresponding frame at the transmitter.
Typically, the process is started by sending one complete frame of the sequence. This frame is referred to as the key frame. After the key frame is sent, successive frames are specified by sending information specifying changes that have occurred relative to the last frame displayed by the receiver. The receiver uses this information to update its frame buffer. If the interframe changes are relatively small, i.e., the scene is almost constant, the available bandwidth will be sufficient to transmit all of the information needed to update the receiver's image. That is, the frame buffer in the receiver will exactly match the corresponding frame of the motion picture at the transmitter.
Unfortunately, the available bandwidth is often insufficient to completely specify all of the interframe changes that have occurred since the last update in 1/30 of a second. Hence, the changes to be sent are prioritized and only the highest priority changes are sent. As a result, errors accumulate in the frame buffer at the receiver. The transmitter keeps track of the magnitude of these errors.
When these errors become sufficiently large, a new key frame must be sent. The time needed to send a key frame is greater than 1/30 of a second. Hence, each time a key frame is sent, the motion will be interrupted for some period of time. For example, at a resolution of 512.times.512 pixels each requiring 8 bits, a key frame would require between one and two seconds to send at 1.44 Mbps, a data rate typically used in teleconference systems. During this time interval additional frames will have been generated at the transmitter. The corrections needed to bring the receiver frame buffer up to date must be sent as a single large update. The combination of the time delay and the large update results in a noticeable loss of continuity in the motion.
The various errors and motion interruptions become particularly objectionable when the sequence of scenes in the motion picture change rapidly. To minimize these problems, either the bandwidth of the communication channel must be increased or the quality of the picture must be reduced. Providing increased bandwidth increases the cost of transmitting the motion picture, and hence, is to be avoided. Similarly, a reduction in picture quality is also to be avoided.
A second problem with this type of prior art motion picture compression system occurs in systems using interlaced video. In interlaced video systems, successive frames are offset relative to one another by an amount to one half the horizontal line spacing in a frame. Hence, the frames do not "line up" precisely when considered as sequence. Prior art systems do not provide an acceptable method for compensating for this motion. As a result, the decompressed motion picture often appears to have jitter.
Broadly, it is an object of the present invention to provide an improved apparatus and method for compressing motion pictures.
It is a further object of the present invention to provide a method and apparatus which is less sensitive to scene changes than prior art systems.
It is a still further object of the present invention to provide a method and apparatus which is more immune to compression artifacts in interlaced video systems than prior art systems.
These and other objects of the present invention will become apparent to those skilled in the art from the following detailed description of the invention and the accompanying drawings.